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DETAILED ACTION 
Claim Objections 

Claim 5 is objected to because of the following informalities: Claim 5 recites 
"The method of streaming data . . ." but does not refer to a previous Claim. Claim 5 is 
interpreted to be an independent Claim and is assumed to mean "A method of 
streaming data . . ." Appropriate correction is required. 

Claim Rejections - 35 USC §112 

Claim 7 contains the trademark/trade name "Real Networks Server". Where a 
trademark or trade name is used in a claim as a limitation to identify or describe a 
particular material or product, the claim does not comply with the requirements of 35 
U.S.C. 112, second paragraph. See Ex parte Simpson, 218 USPQ 1020 (Bd. App. 
1982). The claim scope is uncertain since the trademark or trade name cannot be used 
properly to identify any particular material or product. A trademark or trade name is 
used to identify a source of goods, and not the goods themselves. Thus, a trademark or 
trade name does not identify or describe the goods associated with the trademark or 
trade name. In the present case, the trademark/trade name is used to identify/describe 
a server and, accordingly, the identification/description is indefinite. 
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* 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

Claims 1-2, 4-8, 10-12, and 14-18 rejected under 35 U.S.C. 102(e) as being 
anticipated by Gilbert et al. (U.S. Patent # 6,337,683). 

1 . As for Claim 1 , Gilbert et al. teach a method of streaming a panorama from a server 
to a client, wherein a user can only see the portion of the panorama in a view window 
(see col. 2 lines 54-60 "The present invention simulates movement through multi- 
dimensional space using a series of panoramic images which are projected or displayed 
in sequence. The user's direction of view, that is the selected view window, is 
maintained as the series of images is projected or displayed." The limitation of the user 
seeing only a portion of the panorama is met by col. 5 lines 16-18 "At any particular 
time, only a part (i.e. the view window) from one key frame is visible to a user or 
observer. The direction of each arrow indicates the direction of view, that is, the view 
window or part of the key frame that is projected on a screen for observation." The 
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limitation of the server and client is met by fig. 2. Units 10 (Lens & Electronics and 
Embedded Computer), 20 (Computer that stores images), and 21 (computer that seams 
and stores panoramas) are interpreted to be the server and unit 22 "Viewer" is 
interpreted to be a client, and the user can move the location of the view window in the 
panorama (see col. 5 lines 19-30 The arrows in FIG. 4A are meant to represent a 
particular "view window" from each key frame. As indicated by the change in direction 
of the arrows in the area of FIG. 4A designated by the letter E, a viewer can change his 
direction of view as the pan movie progresses."), said method comprising the steps of: 

dividing the panorama into slices (see col. 7 lines 18-22 "The output of CCD 
array 43a is fed into a JPEG data compression chip 44a. Chip 44a compresses the 
image from lens 41a so that the image can be more easily transmitted to computer 40. 
The output from compression chip 41a fed to an embedded controller 45 which 
transmits signals to computer 40 on a serial time slice basis." It is interpreted that the 
panorama is divided into slice when the signals are transmitted to computer 40 on a 
serial time slice basis), 

transmitting from the server to the client slices of said panorama that contain the 
view window plus a guard band surrounding the view window (See fig. 1 Panorama 3a 
and View Window 3b col. 4 lines 24-30 "FIG. 1 shows a key frame (i.e. a panoramic 
image) or a panorama 3a. Panorama 3a has a view window 3b which corresponds to a 
portion of panorama 3a. Panorama 3a also has associated therewith a number of 
sound tracks 3c." The key frame or Panorama 3a is interpreted to be the guard band 
that is transmitted with the view window 3b), 
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transmitting from the client to the server instructions to change the location of 
said guard band as said user moves said view window (see col. 5 lines 30-38 "The 
sequence of images begins at the point or at the key frame indicated by the letter A and 
the sequence proceeds to the point or key frame indicated by the letter B. At this point 
the viewer can select to either go toward point C or toward point D. The selection may 
be made by "clicking" on a designated "hot spot" in the panorama designated B or it 
may be made depending on some other criteria or action by the user." When a user 
chooses to go towards direction D, it in interpreted that new key frames in the direction 
of point D will be transmitted from the server. Also, when a viewer chooses direction 
point D, the user is inherently moving his view window. The user's action of selecting a 
"hot spot" is interpreted as an instruction sent from the client to the server to change the 
location of said guard band as said user inherently moves said view window.) 

2. As for Claim 2, Gilbert et al. teach said slices are the slices defined in the MPEG 
standard (see col. 3 lines 2-9 "If the images are stored in compressed format, in order to 
conserve time and processing power, when an image is displayed, only the portion of 
the panorama necessary to create a "view window" that is, the portion of the image 
displayed in response to the user's direction of view, is decompressed. Furthermore, 
the images are stored in a format that does not utilize inter-image compression (such as 
that used by the MPEG format)."). 
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4. As for Claim 4, Gilbert et al. teach the client and the server are located on the same 
physical machine (see fig. 9A and col. 9 lines 39-49 "FIG. 9A is a block diagram of the 
viewer 22 which plays or displays pan movies. The main components of the viewer 22 
are a CD disk reader 80, a computer 81 , a display 82, keyboard 84 and a mouse 85. 
Computer 81 reads key frames from disk 80 and displays the view window from each 
key frame on display 82. The operator or user utilizes mouse 85 to indicate a view 
direction. The view direction determines the view window which is displayed on display 
82 by computer 81 .") It is interpreted that the server (Computer 81) and client (Display 
82) are located on the same physical machine. 

5. As for Claim 5, Gilbert et al. teach the method of streaming data relative to a series 
of panoramic images from a server to a client, whereby a view window of said client can 
be displayed to a user (See col. 4 lines 21-30 "A panoramic image provides data 
concerning what is visible in any direction from a particular point in space. At any 
particular time a viewer or user can only look in one direction. The direction or point of 
view of a viewer or user determines the "view window", that is, the part of a panoramic 
image, which is projected on a screen at a particular time."), said method comprising the 
steps of: 

dividing each of said panoramic images into areas (see col. 4 line 27-30 "FIG. 1 
shows a key frame (i.e. a panoramic image) or a panorama 3a. Panorama 3a has a 
view window 3b which corresponds to a portion of panorama 3a." The key frames are 
areas of a panoramic image, 
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streaming a plurality of said areas from each area from said server to said client, 
said plurality of areas including said view window and a guard band around said view 
window (see col. 4 lines 63-67 "Finally the images are compressed as indicated by 
block 21 d and stored with an index file as indicated by block 21 e. Each panorama is 
termed a "key frame". A series of key frames (or more precisely a sequence of view 
windows) projected in sequence is a pan movie."), 

displaying said view window portion of said panorama at said client (See col. 5 
lines 1-8 "For each key frame displayed the viewer 22 determines an appropriate view 
window as indicated by block 22a. The portion of the key frame which corresponds to 
the view window is then de-compressed and displayed as indicated by block 22b."), 

accepting user directions to change the location of said view window (see col. 5 
lines 19-30 "The arrows in FIG. 4A are meant to represent a particular "view window" 
from each key frame. As indicated by the change in direction of the arrows in the area 
of FIG. 4A designated by the letter E, a viewer can change his direction of view as the 
pan movie progresses. It is noted that when a user is viewing a panorama, a user can 
point toward the top or bottom of the screen and thus can view images located in a 360 
degree circle from top to bottom in addition to the horizontal directions illustrated by the 
arrows shown in FIG. 4A." Whenever a user changes his direction of view, his 
corresponding "view window" is going to change. It is interpreted that when a user 
chooses to change his direction of view, instructions to change the location of the view 
window is sent from the client to the server), 
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sending commands to said server to change said plurality of areas being 
streamed to said server when said view window is changed more than a threshold 
amount, and changing the areas streamed from said server to said client in response to 
said commands (see col. 5 lines 19-30 "The arrows in FIG. 4A are meant to represent a 
particular "view window" from each key frame. As indicated by the change in direction 
of the arrows in the area of FIG. 4A designated by the letter E, a viewer can change his 
direction of view as the pan movie progresses. It is noted that when a user is viewing a 
panorama, a user can point toward the top or bottom of the screen and thus can view 
images located in a 360 degree circle from top to bottom in addition to the horizontal 
directions illustrated by the arrows shown in FIG. 4A." Whenever a user changes his 
direction of view, his corresponding "view window" is going to change. It is interpreted 
that when a user chooses to change his direction of view, instructions to change the 
location of the view window is sent from the client to the server. See also col. 9 lines 
31-38 "In order to avoid unnecessary user intervention, "hidden" hot spots may be 
added to connect multiple pan movies. A hidden hotspot is one that does not need to 
be manually selected by the user. With a hidden hot spot, if the user "travels" into a 
particular key frame which has a hidden hot spot, and the user is "looking" in the hot 
spot's general direction, then the system will react based upon the user's implicit 
selection of the hotspot and the user will be sent along the path directed by the hot 
spot." When a user is 'travels 1 and comes to an area where there are hidden hotspots, 
it is interpreted that there is an inherent threshold present that measures how close the 
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a viewer's view window is close to a hotspot wherein the user will be sent along a new 
path that results in the server changing the areas being streamed to the client.) 

6. As for Claim 6, Gilbert et al. teach said areas are MPEG slices (see col. 3 lines 2-9 
"If the images are stored in compressed format, in order to conserve time and 
processing power, when an image is displayed, only the portion of the panorama 
necessary to create a "view window" that is, the portion of the image displayed in 
response to the user's direction of view, is decompressed. Furthermore, the images are 
stored in a format that does not utilize inter-image compression (such as that used by 
the MPEG format).") 

7. As for Claim 7, in light of the 1 12 rejection made above on the limitation of Claim 7, 
Claim 7 is interpreted to refer to any generic server. Gilbert et al. teach a generic server 
(see fig. 2 unit 21 "Computer that Seams and Stores Panoramas" is interpreted to be 
the server). 

8. As for Claim 8, Gilbert et al. teach said panorama is displayed to said user in a 
perspectively correct manner (see col. 13 lines 64-66 "The program given in Appendix A 
will retrieve frames for a move, correct the perspective in accordance with known 
equations and then display the images of the movie in sequence." It is interpreted that 
the panorama displayed to the user will be in a perspectively corrected manner as the 
program in Appendix A will correct the perspective.) 
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10. As for Claim 10, Gilbert et al. teach said server and said client are on the same 
physical machine. See fig. 9A and col. 9 lines 39-49 "FIG. 9A is a block diagram of the 
viewer 22 which plays or displays pan movies. The main components of the viewer 22 
are a CD disk reader 80, a computer 81 , a display 82, keyboard 84 and a mouse 85. 
Computer 81 reads key frames from disk 80 and displays the view window from each 
key frame on display 82. The operator or user utilizes mouse 85 to indicate a view 
direction. The view direction determines the view window which is displayed on display 
82 by computer 81.") It is interpreted that the server (Computer 81) and client (Display 
82) are located on the same physical machine. 

11 . As for Claim 11, the limitations of Claim 1 1 are included in the limitations of Claim 
5. Claim 1 1 is analyzed and rejected as previously presented in Claim 5. Claim 1 1 
further requires means of carrying out the limitations. Gilbert et al. teach the means of 
carrying out the limitation in Appendices A, B, C, and D. See col. 5 lines 9-12 "It is 
noted that the operations indicated by blocks 20a, 20b, 21a to 21e, 22a, 22b, and 22c 
are implemented by means of computer programs which perform the functions shown. 
Computer programs are given in appendices A, B, C, and D." 

12. As for Claim 12, Gilbert et al. teach each of said areas comprise a plurality of 
MPEG slices (see col. 3 lines 2-9 "If the images are stored in compressed format, in 
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order to conserve time and processing power, when an image is displayed, only the 
portion of the panorama necessary to create a "view window" that is, the portion of the 
image displayed in response to the user's direction of view, is decompressed. 
Furthermore, the images are stored in a format that does not utilize inter-image 
compression (such as that used by the MPEG format)."). 

14. As for Claim 14, Gilbert et al. teach said server and said client are on the same 
physical machine (see fig. 9A and col. 9 lines 39-49 "FIG. 9A is a block diagram of the 
viewer 22 which plays or displays pan movies. The main components of the viewer 22 
are a CD disk reader 80, a computer 81, a display 82, keyboard 84 and a mouse 85. 
Computer 81 reads key frames from disk 80 and displays the view window from each 
key frame on display 82. The operator or user utilizes mouse 85 to indicate a view 
direction. The view direction determines the view window which is displayed on display 
82 by computer 81 .") It is interpreted that the server (Computer 81) and client (Display 
82) are located on the same physical machine.). 

15. As for Claim 15, Gilbert et al. teaches all said means are physically located on one 
physical system (See col. 5 lines 9-12 "It is noted that the operations indicated by blocks 
20a, 20b, 21a to 21 e, 22a, 22b, and 22c are implemented by means of computer 
programs which perform the functions shown. Computer programs are given in 
appendices A, B, C, and D.") It is interpreted that the computer programs will be stored 
and run from one computer. 
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16. As for Claim 16, Gilbert et al. teach a system for allowing a series of panoramic 
images stored at a server to be viewed by a user at a client (see col. 2 lines 54-60 "The 
present invention simulates movement through multi-dimensional space using a series 
of panoramic images which are projected or displayed in sequence. The user's 
direction of view, that is the selected view window, is maintained as the series of images 
is projected or displayed."), said system including, 

a streaming server at said server for streaming data to said client (fig. 2 unit 21 
"Computer that Seams and Stores Panoramas" is interpreted to be the server), 

a program at said server for providing said streaming server with an area of 
interest from each panorama to be streamed to said client, said area of interest 
including a view window and a guard band around said view window (see col. 4 lines 
55-67 "The stored images are manually transferred to off line computer 21 which is 
programmed to perform the operations shown in FIG. 3C. First the images are 
decompresses as indicated by block 20a so that they can be manipulated. Next the 
single view images are seamed into a panorama and transformed to equirectangular 
format as indicated by block 21b. Hot spots which indicate break points in a sequence 
of images and sound tracks are added next as indicated by block 21c. Finally the 
images are compressed as indicated by block 21 d and stored with an index file as 
indicated by block 21e. Each panorama is termed a "key frame". A series of key 
frames (or more precisely a sequence of view windows) projected in sequence is a pan 
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movie." Computer 21 prepares the stored panoramic image to be streamed to the 
client.), and 

a program at said client for receiving said data and for selecting the data 
representing said view window and for displaying said view window to said user (see 
col. 5 lines 1-8 "A viewer program in viewer computer 22 is used to view the pan 
movies. The viewer 22 displays in sequence a series of images, that is, a series of key 
frames. For each key frame displayed the viewer 22 determines an appropriate view 
window as indicated by block 22a. The portion of the key frame which corresponds to 
the view window is then de-compressed and displayed as indicated by block 22b."). 

17. As for Claim 17, Gilbert et al. teach a user input device whereby said user can 
move said view window in said area of interest, and a communication path from said 
client to said server whereby said client can instruct said server to change the location 
of said view window (see col. 9 lines 39-49 "FIG. 9A is a block diagram of the viewer 22 
which plays or displays pan movies. The main components of the viewer 22 are a CD 
disk reader 80, a computer 81 , a display 82, keyboard 84 and a mouse 85. Computer 
81 reads key frames from disk 80 and displays the view window from each key frame on 
display 82. The operator or user utilizes mouse 85 to indicate a view direction. 
The view direction determines the view window which is displayed on display 82 
by computer 81. A program which implements blocks 22a to 22c (shown in FIG. 3D) is 
stored in and executed by computer 81." It is interpreted that the user uses the mouse 
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85 to indicate the view direction, which in turn determines the view window, which 
instructs the server to transfer data that is related to the updated frame of view.). 

18. As for Claim 18, Gilbert et al. teach said user input device is a computer mouse. 
See col. 9 lines 44-46 "The operator or user utilizes mouse 85 to indicate a view 
direction. The view direction determines the view window which is displayed on display 
82 by computer 81". 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over Gilbert et 
al. (U.S. Patent # 6,337,683) in view of Driscoll JR (PG Pub 2001/0010555). 

3. As for Claim 3, Gilbert et al. do not expressly teach the streaming from the server to 
the client is handled by a streaming server and a plug-in to said server provides the 
slices in said guard band. However, in the same field of endeavor, Driscoll JR teaches 
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a method of capturing a 360 degree panoramic image and a method of displaying a 
portion of the panoramic image at a client is handled by a streaming server and a plug- 
in to said server. See Driscoll JR Fig. 13A and [0106] "Referring back to FIG. 13a, user 
input processing routines 1250 and 1253 processing the user's commands. When a 
user requests a viewpoint change, the new viewpoint is communicated to the respective 
annular to video conversion units 1240 or 1243 such that it will begin converting images 
from the new user viewpoint. In an alternate embodiment, the user input processing 
routines are place within the client program on the client computer system. For 
example, in an embodiment of a WWW browser program, a plug-in program can 
process the user commands and simply pass the location of the video viewport to the 
server." In light of the teaching of Driscoll JR, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify the teaching of 

* 

Gilbert et al. to have a streaming server and plug-in to said server provide the slices in 
said guard band to the client. One of ordinary skill in the art at the time the invention 
was made would have been motivated to do this if the panoramic image was streamed 
via a WWW browser program that requires a plug-in. 

Claims 9 and 13 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Gilbert et al. (U.S. Patent # 6,337,683) in view of Driscoll et al. (U.S. Patent # 
6,219,089) 



1 
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9. As for Claims 9 and 13, Gilbert et al. do not expressly teach said server 
simultaneously streams portions of different panoramas to different clients. However, in 
the same field of endeavor, Driscoll et al. teach a method and apparatus of distributing 
portions of panoramic images to a plurality of clients. See Driscoll et al. col. 6 lines 37- 
53 "The clients in panoramic client server environment would likely consist of 
workstations and personal computers. The capabilities of each different panoramic 
client would vary depending upon the client systems specifications including the 
computer make, the processor type, the processor generation, the amount of random 
access memory available, and the bus speed." In light of the teaching of Driscoll et al., 
it would have been obvious to one of ordinary skill in the art at the time the invention 
was made to have modified the teaching of Gilbert et al. to have multiple clients. It is 
interpreted that when there are multiple clients, the server will simultaneously streams 
portions of different panoramas to different clients. One of ordinary skill in the art at the 
time the invention was made would have been motivated to do this # in order to service 
multiple clients using a single server in order to reach more clients. 



Claims 19 and 20 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Gilbert et al. (U.S. Patent # 6,337,683). 

19. As for Claim 19, Gilbert et al. do not expressly teach said panoramic images are 
stored at said server using MPEG compression forming "I" and "B" or "P" frames. 
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Gilbert et al. do however teach said panoramic images are stored using the MPEG 
compression format (see Gilbert et al. col. 3 lines 7-9 "Furthermore, the images are 
stored in a format that does not utilize inter-image compression (such as that used by 
the MPEG format)"). However, Official Notice (MPEP § 2144.03) is taken that both the 
concepts and advantages of using MPEG compression forming "I" (intra- frames) and 
"B" (Bi-direction predicted frames) or "P" (forward-predicted frames) are well known and 
expected in the art. At the time the invention was made, it would have been obvious to 
one with ordinary skill in the art to have the MPEG compression be constructed from "I" 
and "B" frames or "I" and "P" frames. One of ordinary skill in the art at the time the 
invention was made would have been motivated to use "I" and "B" frames if bi- 
directional frame prediction is desired and T and T" frames if bi-directional frame 
prediction is not desired. 

20. As for Claim 20, the modified Gilbert et al. teaches the region of interest from T' 
frames and entire "B" or "P" frames are transmitted from said server to said client (see 
col. 4 lines 40-46 "The images are transferred from computer 20 to off line computer 21 . 
Computer 21 seams the images into panoramas, transforms the images to 
equirectangular format, adds other information to the images, compresses the 
panoramas, and links the panoramas into a pan movie. Finally the pan movie is viewed 
on viewer 22."). 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Kirubel Aklilu whose telephone number is 571-272- 
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